This application is generally directed to Web service black box testing and, more particularly, to synthesizing effective payloads for black box testing of Web services.
Web services are a central component in the design of Web systems. A Web service may provide remote online functionality to Web applications and other Web services. Automated black box testing of Web services for functional problems (for example, security vulnerabilities) is a challenge, even compared to black box testing of Web applications. A key reason for this is that the return value of a Web method in a Web Service is typically a primitive value like an integer or a Boolean (though in principle the Web method may return an arbitrary object). This is unlike a Web application iteration where the returned result of an Internet hypertext transfer protocol (HTTP) request is an HTTP response that can be analyzed to determine whether the test payload was successful.
A significant gap in expressiveness between an HTTP response and a primitive value hinders the applicability of black box validation techniques from testing of Web applications to Web services. In sum, there is simply not enough information in the result of a Web service to make any educated or principled estimation about the success of an attack. A classic example of this, from the space of security testing, is checking for cross-site scripting (XSS) vulnerabilities, where Web application validation checks whether an input payload (a script) is contained in a hypertext markup language (HTML) document returned by the Web application. The same test cannot be performed if the returned value is a number as is likely for a Web service.
The only way of getting sufficient feedback from a Web service, such that black box validation can be applied, is if the Web service is driven into an error condition, in which case its return value is normally an error message that often contains data from the input payload. The data published as part of the response can then be subjected to validation, for example, by looking for the input script in the error message.
Naturally, however, the error message would be considered interesting only if it is due to an illegal behavior within the business logic of the Web service. Otherwise, if the cause of the exception is a superficial validation step in the outer layers of the Web service, where the scanner has not exercised the “true” functionality of the Web service, testing is shallow and incomplete. For example, such an exception can occur during parsing of an incoming input message or when checking whether a number is within legal bounds.